install-buildtools: support buildtools-make-tarball and update to 4.1
PATCH 1/4 install-buildtools: support buildtools-make-tarball and update to 4.1
From: Paul Eggleton <[email protected]>
Support installing buildtools-make-tarball that is built in version 4.1 and later for build hosts with a broken make version. Also update the default version values to 4.1.
--- scripts/install-buildtools | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)
scripts/install-buildtools +16 -4
@@ -57,8 +57,8 @@ logger = scriptutils.logger_create(PROGNAME, stream=sys.stdout) DEFAULT_INSTALL_DIR = os.path.join(os.path.split(scripts_path)[0],'buildtools') DEFAULT_BASE_URL = 'http://downloads.yoctoproject.org/releases/yocto' -DEFAULT_RELEASE = 'yocto-3.4' -DEFAULT_INSTALLER_VERSION = '3.4' +DEFAULT_RELEASE = 'yocto-4.1' +DEFAULT_INSTALLER_VERSION = '4.1' DEFAULT_BUILDDATE = '202110XX' # Python version sanity check
@@ -154,6 +154,8 @@ def main(): group.add_argument('--without-extended-buildtools', action='store_false', dest='with_extended_buildtools', help='disable extended buildtools (traditional buildtools tarball)') + group.add_argument('--make-only', action='store_true', + help='only install make tarball') group = parser.add_mutually_exclusive_group() group.add_argument('-c', '--check', help='enable checksum validation', default=True, action='store_true')
@@ -170,6 +172,9 @@ def main(): args = parser.parse_args() + if args.make_only: + args.with_extended_buildtools = False + if args.debug: logger.setLevel(logging.DEBUG) elif args.quiet:
@@ -197,7 +202,10 @@ def main(): if not args.build_date: logger.error("Milestone installers require --build-date") else: - if args.with_extended_buildtools: + if args.make_only: + filename = "%s-buildtools-make-nativesdk-standalone-%s-%s.sh" % ( + arch, args.installer_version, args.build_date) + elif args.with_extended_buildtools: filename = "%s-buildtools-extended-nativesdk-standalone-%s-%s.sh" % ( arch, args.installer_version, args.build_date) else:
@@ -207,6 +215,8 @@ def main(): buildtools_url = "%s/milestones/%s/buildtools/%s" % (base_url, args.release, safe_filename) # regular release SDK else: + if args.make_only: + filename = "%s-buildtools-make-nativesdk-standalone-%s.sh" % (arch, args.installer_version) if args.with_extended_buildtools: filename = "%s-buildtools-extended-nativesdk-standalone-%s.sh" % (arch, args.installer_version) else:
@@ -303,7 +313,9 @@ def main(): if args.with_extended_buildtools and not m: logger.info("Ignoring --with-extended-buildtools as filename " "does not contain 'extended'") - if args.with_extended_buildtools and m: + if args.make_only: + tool = 'make' + elif args.with_extended_buildtools and m: tool = 'gcc' else: tool = 'tar'
Signature · 1 line
2.25.1
PATCH 2/4 install-buildtools: remove md5 checksum validation
From: Aleksandar Nikolic <[email protected]>
No need to validate with the md5 checksum, as the file is not even uploaded to the Yocto release webpage (the download never failed due to a wrong indentation of an else statement). For validation purposes, use the sha256 checksum only.
--- scripts/install-buildtools | 27 ++++++++++----------------- 1 file changed, 10 insertions(+), 17 deletions(-)
scripts/install-buildtools +10 -17
@@ -238,19 +238,15 @@ def main(): # Verify checksum if args.check: logger.info("Fetching buildtools installer checksum") - checksum_type = "" - for checksum_type in ["md5sum", "sha256sum"]: - check_url = "{}.{}".format(buildtools_url, checksum_type) - checksum_filename = "{}.{}".format(filename, checksum_type) - tmpbuildtools_checksum = os.path.join(tmpsdk_dir, checksum_filename) - ret = subprocess.call("wget -q -O %s %s" % - (tmpbuildtools_checksum, check_url), shell=True) - if ret == 0: - break - else: - if ret != 0: - logger.error("Could not download file from %s" % check_url) - return ret + checksum_type = "sha256sum" + check_url = "{}.{}".format(buildtools_url, checksum_type) + checksum_filename = "{}.{}".format(filename, checksum_type) + tmpbuildtools_checksum = os.path.join(tmpsdk_dir, checksum_filename) + ret = subprocess.call("wget -q -O %s %s" % + (tmpbuildtools_checksum, check_url), shell=True) + if ret != 0: + logger.error("Could not download file from %s" % check_url) + return ret regex = re.compile(r"^(?P<checksum>[0-9a-f]+)\s+(?P<path>.*/)?(?P<filename>.*)$") with open(tmpbuildtools_checksum, 'rb') as f: original = f.read()
@@ -263,10 +259,7 @@ def main(): logger.error("Filename does not match name in checksum") return 1 checksum = m.group('checksum') - if checksum_type == "md5sum": - checksum_value = md5_file(tmpbuildtools) - else: - checksum_value = sha256_file(tmpbuildtools) + checksum_value = sha256_file(tmpbuildtools) if checksum == checksum_value: logger.info("Checksum success") else:
Signature · 1 line
2.25.1
PATCH 3/4 install-buildtools: fix "test installation" step
In reply to Aleksandar NikolicFrom: Aleksandar Nikolic <[email protected]>
The "Test installation" step fails with some harmless error messages (see [1]). This can however make a user think that the buildtools have not been installed correctly.
Two reasons for the error messages:
- some envvars in the environment-setup-<arch>-pokysdk-linux file start and end with double quotes (e.g., PATH) and are as such written into python os.environ. This leads that their usage is not valid later when testing the installation. This patch removes the double quotes before writing, if they are present.
- if installation directory (install_dir), given through the option --directory, is given as a relative path, checking if the path to a tool (e.g., gcc) in buildtools starts it will always fail. This patch converts the install_dir variable to an absolute path.
[1]
ERROR: Something went wrong: tar not found in ./build-tools
ERROR: Something went wrong: installation failed
--- scripts/install-buildtools | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
scripts/install-buildtools +12 -2
@@ -102,6 +102,16 @@ def sha256_file(filename): import hashlib return _hasher(hashlib.sha256(), filename) +def remove_quotes(var): + """ + If a variable starts and ends with double quotes, remove them. + Assumption: if a variable starts with double quotes, it must also + end with them. + """ + if var[0] == '"': + var = var[1:-1] + return var + def main(): global DEFAULT_INSTALL_DIR
@@ -273,7 +283,7 @@ def main(): os.chmod(tmpbuildtools, st.st_mode | stat.S_IEXEC) logger.debug(os.stat(tmpbuildtools)) if args.directory: - install_dir = args.directory + install_dir = os.path.abspath(args.directory) ret = subprocess.call("%s -d %s -y" % (tmpbuildtools, install_dir), shell=True) else:
@@ -294,7 +304,7 @@ def main(): if match: env_var = match.group('env_var') logger.debug("env_var: %s" % env_var) - env_val = match.group('env_val') + env_val = remove_quotes(match.group('env_val')) logger.debug("env_val: %s" % env_val) os.environ[env_var] = env_val
Signature · 1 line
2.25.1
PATCH 4/4 scripts/install-buildtools: Update to 4.0.21
In reply to Aleksandar NikolicFrom: Aleksandar Nikolic <[email protected]>
Update to the 4.0.21 release of the 4.0 series for buildtools.
--- scripts/install-buildtools | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
scripts/install-buildtools +3 -3
@@ -56,9 +56,9 @@ PROGNAME = 'install-buildtools' logger = scriptutils.logger_create(PROGNAME, stream=sys.stdout) DEFAULT_INSTALL_DIR = os.path.join(os.path.split(scripts_path)[0],'buildtools') -DEFAULT_BASE_URL = 'http://downloads.yoctoproject.org/releases/yocto' -DEFAULT_RELEASE = 'yocto-4.1' -DEFAULT_INSTALLER_VERSION = '4.1' +DEFAULT_BASE_URL = 'https://downloads.yoctoproject.org/releases/yocto' +DEFAULT_RELEASE = 'yocto-4.0.21' +DEFAULT_INSTALLER_VERSION = '4.0.21' DEFAULT_BUILDDATE = '202110XX' # Python version sanity check
Signature · 1 line
2.25.1
Re: [OE-core] [PATCH 4/4] scripts/install-buildtools: Update to 4.0.21
Hi Aleksandar,
Aleksandar Nikolic via lists.openembedded.org <aleksandar.nikolic010= [email protected]> escreveu (segunda, 23/09/2024 à(s) 10:00):
Quoted 15 lines
From: Aleksandar Nikolic <[email protected]>
Update to the 4.0.21 release of the 4.0 series for buildtools.
Signed-off-by Aleksandar Nikolic <[email protected]>--- scripts/install-buildtools | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)scripts/install-buildtools +0
-0@@ -56,9 +56,9 @@ PROGNAME = 'install-buildtools' logger = scriptutils.logger_create(PROGNAME, stream=sys.stdout)
scripts/install-buildtools PROGNAME = 'install-buildtools' · line 63
Quoted 10 lines
scripts/install-buildtools +3
-3DEFAULT_INSTALL_DIR = os.path.join(os.path.split(scripts_path)[0],'buildtools') -DEFAULT_BASE_URL = 'http://downloads.yoctoproject.org/releases/yocto' -DEFAULT_RELEASE = 'yocto-4.1' -DEFAULT_INSTALLER_VERSION = '4.1' +DEFAULT_BASE_URL = 'https://downloads.yoctoproject.org/releases/yocto' +DEFAULT_RELEASE = 'yocto-4.0.21' +DEFAULT_INSTALLER_VERSION = '4.0.21'
This first patch in the series updates this version to 4.1. What is the reason to downgrade this now on the last patch?
Jose
Quoted 18 lines
scripts/install-buildtools +0
-0DEFAULT_BUILDDATE = '202110XX'# Python version sanity check
--
2.25.1-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#204799):
https://lists.openembedded.org/g/openembedded-core/message/204799
Mute This Topic: https://lists.openembedded.org/mt/108604377/5052612
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Best regards, José Quaresma
Re: [PATCH 4/4] scripts/install-buildtools: Update to 4.0.21
Hi Jose,
Quoted 23 lines · Jose Quaresma
Hi Aleksandar,
Aleksandar Nikolic via lists.openembedded.org (
http://lists.openembedded.org ) <aleksandar.nikolic010= [email protected]Quoted 18 lines
escreveu (segunda, 23/09/2024 à(s) 10:00):
From: Aleksandar Nikolic < [email protected] >
Update to the 4.0.21 release of the 4.0 series for buildtools.
Signed-off-by Aleksandar Nikolic < [email protected] >---
scripts/install-buildtools| 6 +++---1 file changed, 3 insertions(+), 3 deletions(-)scripts/install-buildtools +0
-0@@ -56,9 +56,9 @@ PROGNAME = 'install-buildtools' logger = scriptutils.logger_create(PROGNAME, stream=sys.stdout)
Quoted 14 lines
scripts/install-buildtools PROGNAME = 'install-buildtools' · line 61
Quoted 8 lines
DEFAULT_INSTALL_DIR= os.path.join(os.path.split(scripts_path)[0],'buildtools')scripts/install-buildtools +3
-3-DEFAULT_BASE_URL = ' http://downloads.yoctoproject.org/releases/yocto ' -DEFAULT_RELEASE = 'yocto-4.1' -DEFAULT_INSTALLER_VERSION = '4.1' +DEFAULT_BASE_URL = ' https://downloads.yoctoproject.org/releases/yocto ' +DEFAULT_RELEASE = 'yocto-4.0.21' +DEFAULT_INSTALLER_VERSION = '4.0.21'This first patch in the series updates this version to 4.1. What is the reason to downgrade this now on the last patch?
I was thinking since 4.1 (and higher versions) are EOL and since this is the kirkstone branch, the buildtools should also reflect the latest available release on the kirkstone branch, which is 4.0.21. But I am not really sure what's the right solution here, if someone else could chime in, I'd be thankful.
Aleksandar
Quoted 23 lines
Jose
Quoted 9 lines
DEFAULT_BUILDDATE= '202110XX'# Python version sanity check
--
2.25.1--
Best regards,José Quaresma
Re: [OE-core] [PATCH 4/4] scripts/install-buildtools: Update to 4.0.21
Quoted 1 line · Aleksandar Nikolic via lists.openembedded.org
I was thinking since 4.1 (and higher versions) are EOL and since this is the kirkstone branch, the buildtools should also reflect the latest available release on the kirkstone branch, which is 4.0.21. But I am not really sure what's the right solution here, if someone else could chime in, I'd be thankful.
If these patches are for a stable branch then please make that obvious by putting [kirkstone] in the subject.
Ross
Re: [PATCH 4/4] scripts/install-buildtools: Update to 4.0.21
Quoted 4 lines · Ross Burton
On 24 Sep 2024, at 08:57, Aleksandar Nikolic via lists.openembedded.org <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
I was thinking since 4.1 (and higher versions) are EOL and since this is the kirkstone branch, the buildtools should also reflect the latest available release on the kirkstone branch, which is 4.0.21. But I am not really sure what's the right solution here, if someone else could chime in, I'd be thankful.
If these patches are for a stable branch then please make that obvious by putting [kirkstone] in the subject.
I forgot to do that and only saw it after the patches were sent out (did it correctly for scarthgap later). Should I send a new patch series (v2) with [kirkstone] in the subject?
Aleksandar
Quoted 2 lines
Ross